Structured variable selection in support vector machines

When applying the support vector machine (SVM) to high-dimensional classification problems, we often impose a sparse structure in the SVM to eliminate the influences of the irrelevant predictors. The lasso and other variable selection techniques have been successfully used in the SVM to perform automatic variable selection. In some problems, there is a natural hierarchical structure among the variables. Thus, in order to have an interpretable SVM classifier, it is important to respect the heredity principle when enforcing the sparsity in the SVM. Many variable selection methods, however, do not respect the heredity principle. In this paper we enforce both sparsity and the heredity principle in the SVM by using the so-called structured variable selection (SVS) framework originally proposed in Yuan, Joseph and Zou (2007). We minimize the empirical hinge loss under a set of linear inequality constraints and a lasso-type penalty. The solution always obeys the desired heredity principle and enjoys sparsity. The new SVM classifier can be efficiently fitted, because the optimization problem is a linear program. Another contribution of this work is to present a nonparametric extension of the SVS framework, and we propose nonparametric heredity SVMs. Simulated and real data are used to illustrate the merits of the proposed method.Comment: Published in at http://dx.doi.org/10.1214/07-EJS125 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Structured variable selection and estimation

In linear regression problems with related predictors, it is desirable to do variable selection and estimation by maintaining the hierarchical or structural relationships among predictors. In this paper we propose non-negative garrote methods that can naturally incorporate such relationships defined through effect heredity principles or marginality principles. We show that the methods are very easy to compute and enjoy nice theoretical properties. We also show that the methods can be easily extended to deal with more general regression problems such as generalized linear models. Simulations and real examples are used to illustrate the merits of the proposed methods.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS254 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Optimization Framework and Graph-Based Approach for Relay-Assisted Bidirectional OFDMA Cellular Networks

This paper considers a relay-assisted bidirectional cellular network where the base station (BS) communicates with each mobile station (MS) using OFDMA for both uplink and downlink. The goal is to improve the overall system performance by exploring the full potential of the network in various dimensions including user, subcarrier, relay, and bidirectional traffic. In this work, we first introduce a novel three-time-slot time-division duplexing (TDD) transmission protocol. This protocol unifies direct transmission, one-way relaying and network-coded two-way relaying between the BS and each MS. Using the proposed three-time-slot TDD protocol, we then propose an optimization framework for resource allocation to achieve the following gains: cooperative diversity (via relay selection), network coding gain (via bidirectional transmission mode selection), and multiuser diversity (via subcarrier assignment). We formulate the problem as a combinatorial optimization problem, which is NP-complete. To make it more tractable, we adopt a graph-based approach. We first establish the equivalence between the original problem and a maximum weighted clique problem in graph theory. A metaheuristic algorithm based on any colony optimization (ACO) is then employed to find the solution in polynomial time. Simulation results demonstrate that the proposed protocol together with the ACO algorithm significantly enhances the system total throughput.Comment: 27 pages, 8 figures, 2 table